Spectral improvement of digital camera color images

ABSTRACT

Systems and methods for modifying a color image of a scene are provided. The systems and methods involve illuminating at least a portion of the scene with a light having a known spectral power distribution, and detecting a finite number of spectral components of light received from the scene under such illumination. The detected values of spectral components of light received from the scene are used to modify the color image or rendition of the scene.

TECHNICAL FIELD

The present application relates, in general, to color imagingtechnology. In particular, the application relates to the spectralcontent of images.

BACKGROUND

Color is a psychological phenomenon based on the interaction of thespectrum of light (distribution of light energy versus wavelength) withlight receptors in the eye, which have specific sensitivities forspecific spectral wavelength bands or regions (hereinafter “spectralcomponents”) of light. Color categories and physical specifications ofcolor are also associated with objects, materials, light sources, etc.,based on their physical properties such as light absorption, reflection,or emission spectra.

In the physical sense, a color is the ratio of intensities measured indifferent spectral components of the electromagnetic spectrum. In thephysiological sense, in humans, the different spectral components aredefined by the spectral sensitivity curves of three different types ofreceptors in the eye (i.e., the so-called red, blue, and green cones).The human brain processes and combines signals from the red, blue andgreen cones to create a composite impression or image of a scene. Allcolors in the scene are perceived as combinations of the red, blue andgreen cone signals. The range or gamut of colors that are perceived byhumans is represented, for example, by the CIE 1931 chromaticity diagram(FIG. 5).

Man-made color image sensors (e.g., color film, or digital cameras usingCCD or CMOS sensors) also sense light intensities in a finite number ofdistinct spectral components. Various types of color image sensorsdiffer in how they separate and measure the distinct spectralcomponents. For example, a color film may have a stack of threedifferent emulsion layers that are exposed by red, green, and bluecomponents of light, respectively. A digital camera may use an array oflayered sensors so that every pixel, like a color film, contains a stackof sensors sensitive to individual colors (e.g., sensor available underthe Foveon trademark). More commonly, digital cameras use a spatialcolor filter array (e.g., a Bayer filter) positioned on top of a CMOS orCCD sensor to capture different spectral components of light incorresponding nominal pixel types.

The man-made color image sensors sense and collect intensity data foreach distinct spectral component received from a scene. The data foreach spectral component is monochrome, i.e., it includes only intensityinformation, but no color information. To create an approximate colorimage or rendition of a scene, the different intensity data for thedistinct spectral components are processed, encoded to certain colorsand intensities, and combined.

The color image or rendition is an approximation, at least in part,because the collected intensity data does not include full or completeinformation on the spectral distribution of light received from thescene. The intensity data on the spectral distribution of light receivedfrom the scene is limited by, for example, the finite number of spectralcomponents sensed, the shape and the widths of the sensor spectralsensitivities for each spectral component, and overlap of the spectralsensitivities for the different spectral components. FIG. 3 shows thered, blue and green spectral responses of an exemplary CMOS sensor witha RGB color filter. Consideration is now being given to ways ofincreasing or supplementing the color image data with more informationon the spectral distribution of light received from the scene.

SUMMARY

Systems, apparatuses, and methods for modifying the spectral informationcontent of color images are provided.

An exemplary system for modifying a spectral response representation ofa scene obtained by an imaging device includes an apparatus having asupplemental illumination device and/or data processing circuitry.

The supplemental illumination device is operable to illuminate the scenewith light having a spectral distribution for which at least somequalifying information is available (e.g., “a known spectraldistribution”).

The data processing circuitry renders a color image of the scene byprocessing a plurality of sets of spectral components of light receivedfrom the scene under a corresponding plurality of scene illuminationsincluding at least an illumination with light having a known spectraldistribution, respectively. The rendered color image of the scene may bebased at least in part on the detected spectral components of lightreceived from the scene under the scene illuminations, thecharacteristic spectral response functions R (λ) of the imaging device,and the known spectral distribution of light in at least anillumination.

Methods for improving a color image or rendition of a scene may utilizeadditional scene information developed by illuminating the scene withlight having a known spectral distribution.

An exemplary method relates to improving a color image or scenerendition, which may have been obtained under a first sceneillumination. The method involves illuminating at least a portion of thescene in a second scene illumination with a light component having aknown spectral power distribution, detecting a finite number of spectralcomponents of light received from the scene under such illumination, andmodifying the color image or rendition of the scene according to thedetected values of the finite number of spectral components of lightreceived from the scene under such illumination.

Another exemplary method relates to rendering a color image of a scene.The method involves processing a plurality of sets of values of spectralcomponents of light received from the scene under a correspondingplurality of scene illuminations including a first and a second sceneillumination. The method further includes rendering the color image ofthe scene based at least in part on the values of the spectralcomponents of light received from the scene under the first and secondscene illuminations, the characteristic spectral response functions ofan imaging device that was used to detect of the spectral components oflight received from the scene, and the known spectral distribution oflight in the second scene illumination.

The foregoing summary is illustrative only and is not intended to belimiting. In addition to the illustrative aspects, embodiments, andfeatures described above, further aspects, embodiments, and features ofthe solutions will become apparent by reference to the drawings and thefollowing detailed description.

BRIEF DESCRIPTION OF THE FIGURES

In the accompanying drawings:

FIG. 1 is a block diagram illustrating an exemplary system for acquiringa color image or rendition of a scene, in accordance with the principlesof the solutions described herein;

FIG. 2 is a schematic illustration of exemplary spectral distributionshapes of light that may be used to illuminate a scene, in accordancewith the principles of the solutions described herein;

FIG. 3 is a block diagram illustrating an exemplary apparatus formodifying a spectral response representation of a scene obtained by animaging device, in accordance with the principles of the solutionsdescribed herein;

FIG. 4 is a graphical illustration of red, green and blue spectralresponses of an exemplary CMOS sensor with a RGB color filter;

FIG. 5 is a reproduction of the 1931 CIE chromaticity diagram, which isa standard representation of the color gamut of human vision;

FIG. 6 is a flow diagram illustrating an exemplary method for improvinga color image or scene rendition, in accordance with the principles ofthe solutions described herein;

FIG. 7 is a flow diagram illustrating an exemplary method for renderinga color image of a scene in response to control or command signal, inaccordance with the principles of the solutions described herein; and

FIG. 8 is a flow diagram illustrating an exemplary method for adjustinga characteristic spectral response function of an “in use” digitalimaging device, in accordance with the principles of the solutionsdescribed herein.

Throughout the figures, unless otherwise stated, the same referencenumerals and characters are used to denote like features, elements,components, or portions of the illustrated embodiments.

DETAILED DESCRIPTION

In the following detailed description, reference is made to theaccompanying drawings, which form a part hereof. In the drawings,similar symbols typically identify similar components, unless contextdictates otherwise. The illustrative embodiments described in thedetailed description, drawings, and claims are not meant to be limiting.Other embodiments may be utilized, and other changes may be made,without departing from the spirit or scope of the subject matterpresented here.

The terms “image” and “rendition of a scene” as used herein will beunderstood to include, without limitation, reference to image and scenerendition data and/or signals, respectively. The terms may also, incontext, refer to display or output of such data and/or signals. Moregenerally, any nominal term used herein to refer to a physical,mathematical, conceptual, perceptual, abstract object or aspect (e.g.,spectral power distribution, spectral components of light, spectralresponse functions, etc.) will be understood to include, withoutlimitation, reference to data and/or signals, or other tangiblerepresentations of the named object or aspect.

A system for acquiring spectral characteristics of a scene under a firstscene illumination includes an imaging device configured to detect afinite number (N) of spectral components of light received from thescene, a supplemental illumination device operable to provide a secondscene illumination with light having a known spectral distribution, andprocessing circuitry to produce a signal indicative of a spectral powerdistribution response characteristic of the scene. The processingcircuitry is responsive to the detected spectral components of lightreceived from the scene under the first and second scene illuminations,characteristic spectral response functions R_(n)(λ), n=1, . . . N of theimaging device, and the known spectral distribution of the lightprovided by the supplemental illumination device.

FIG. 1 shows an exemplary system 100 for acquiring a color image orrendition of a scene 180 under a first scene illumination. The firstscene illumination may include ambient and/or artificial lightingprovided by an optional artificial lighting unit 130. Light 182 receivedfrom the scene (e.g., light reflected, transmitted, or emitted by thescene) may have a spectral power distribution (SPD(λ)), which ischaracteristic of the scene.

Exemplary system 100 includes an imaging device 110, a supplementalillumination device 140, and optional control and processing circuitry(e.g., controller 160 and data processor 170).

Imaging device 110 is configured to obtain color information from light182 received from the scene via suitable optics (e.g., optics 150).Imaging device 110 may include an image sensor 120 (e.g., a solid stateCCD or CMOS device having an array of pixels), which can sense light andconvert it into data and/or signals. Further, image sensor 120 may beconfigured to sense a finite number N of spectral components of light182 received from scene 180. For example, image sensor 120 may becoupled to a color filter array (e.g., a Bayer filter that passes red,green, or blue light to selected pixels in the sensor) to form spatiallyinterlaced pixel grids that are each sensitive to a respective one ofthe N spectral components (e.g., red, green, and blue, respectively).Alternatively, each pixel of image sensor 120 may include a stack ofsensing elements in which each sensing element is sensitive to arespective one of the N spectral components of light 182. Further, anembodiment of imaging device 110 may include a plurality of imagesensors 120 with each sensor configured to sense a respective one of theN spectral components of light 182. For this embodiment, optics 150 mayinclude, for example, a dichroic prism arrangement (not shown) toseparate the different spectral components of light 182. In any case,imaging device 110 may have characteristic spectral response functionsR_(n)(λ), n=1, . . . N, for sensing the N spectral components of light182. (See e.g., FIG. 4). The wavelengths of the N spectral componentsmay correspond to primary colors that define a color gamut of theimaging device. In some instances, two or more of the imaging device'scharacteristic spectral response functions R_(n)(λ), n=1, . . . N, whichrespectively correspond to the N detected spectral components maysubstantially overlap.

In system 100, imaging device 110 is operationally coupled tosupplemental illumination device 140. The latter is operable toilluminate scene 180 in a second scene illumination with light 142having a spectral distribution for which at least some qualifyinginformation (e.g., wavelength(s) and/or relative intensity) is available(hereinafter “a known spectral distribution”). For example, supplementalillumination device 140 may provide substantially monochromatic light142 at the one or more wavelength(s) λ_(k), k=1, . . . K forilluminating scene 180. The monochromatic light may, for example, befiltered light, diffraction grating separated light, or laser light.Alternatively or additionally, light 142 may include light with otherspectral distribution shapes. FIG. 2 shows exemplary spectraldistributions of light 142 formed by using wideband, narrowband, andCIE-based filter sets (146).

The operations of system 100 components (e.g., imaging device 110 andsupplemental illumination device 140) may be controlled or co-ordinatedby the optional control circuitry (e.g., controller 160). Further, theoptional processing circuitry (e.g. data processor 170) in system 100may be configured to receive, store, modify and/or process data and/orsignals related to system 100 components and their operations including,for example, image or scene data sensed by image sensor 120, and data onthe second scene illumination provided by supplemental illuminationdevice 140. Elements or portions of the control circuitry and/or theprocessing circuitry may be disposed at any system 100 component (e.g.,imaging device 110, and supplemental illumination device 140), at one ormore external or remote locations, or distributed between system 100components and the external locations.

In one embodiment of system 100, imaging device 110 is configured tosense the spectral components of light 182 received from scene 180 underfirst scene illumination and second scene illuminations. Further, inthis embodiment, the processing circuitry may be configured to beresponsive to signals and/or data corresponding to the sensed spectralcomponents of light 182 received from scene 180 under the first andsecond scene illuminations, the characteristic spectral responsefunctions of the imaging device, and the known spectral distribution ofthe light provided by the supplemental illumination device. Theprocessing circuitry may be configured to process live imaging devicedata, or optionally to process such data at a later time (e.g., batchprocess such data).

A response of the processing circuitry may, for example, includecomputing signals and/or data corresponding to the SPD of scene 180based at least in part on the sensed spectral components under the firstscene illumination, and computing signals and/or data corresponding to acorrected or modified SPD based at least in part on additional spectralinformation in the sensed spectral components under the second sceneillumination.

The processing circuitry may store, display, or transmit the computeddata and/or signals to other processors or users. The computed SPD dataand/or signals may be utilized to make a visual color image or renditionof the scene.

With respect to the various scene illuminations, supplementalillumination device 140 may be operable so that in the second sceneillumination light with the known spectral distribution is added to thelight of the first scene illumination. Alternatively, in the secondscene illumination, light with the known spectral distribution mayreplace the light of the first scene illumination. More generally,system 100 may be configured so that a first of a plurality the sceneilluminations includes ambient and/or artificial light provided byoptional lighting unit 130, and supplemental illumination device 140 maybe configured so that the first of the plurality of scene illuminationsadditionally includes the light with the known spectral distribution anda second of the plurality of scene illuminations includes only the lightwith the known spectral distribution.

Supplemental illumination device 140 may include any suitable number andtype light sources (e.g., solid state lighting (SSL), light emittingdiodes (LEDs), lasers, gas discharge lamps, incandescent lamps,fluorescent lamps, High Intensity Discharge (HID) lamps, gas dischargelamps, arc lamps, and combustion lamps). Suitable filter assemblies maybe deployed to shape or select the spectral distribution of the emittedlight and to direct the light toward scene 180. The light sources mayinclude one or more substantially monochromatic light sources. Anexemplary supplemental illumination device 140 includes an array of oneor more light sources emitting light at one or more wavelength(s) λ_(k),k=1, . . . K, which may correspond to one or more different colors R, G,B, Y, C, M, . . . , respectively. It should be noted that in someapproaches two or more light sources may correspond to a given “color.”For example, in some approaches the supplemental illumination mayinclude two or more green light sources and/or two or more blue lightsources, or any other combination of light sources. Each light sourcecorresponding to a given color may have a spectral distributiondifferent from that of one or more other light sources corresponding tothe given color. For example, a first of the two or more green lightsources may have a line center frequency different from a second of thetwo or more green light sources. Similarly, a first of the two or moregreen light sources may have a frequency width or range different from asecond of the two or more green light sources.

Further, supplemental illumination device 140 may optionally includeoptics configured to shape or direct the supplemental light toward thescene. An exemplary supplemental illumination device 140 includesadaptive optics (e.g., an array of lenses or microlenses 144) coupled tothe one or more light sources. The adaptive optics may be configured todirect light having a known spectral distribution onto at least aportion of the scene. Moreover, the adaptive optics may distributesupplemental light from supplemental illumination device 140 accordingto a selected distribution (e.g., a spatial distribution). For example,in one approach the adaptive optics may distribute supplementalillumination substantially uniformly across a portion of the field ofview or may provide a gradient of intensity across a portion of thefield of view. Additionally, the adaptive optics may vary fromillumination to illumination, for example, responsive to the controlcircuitry and/or the processing circuitry. For example, during a firstscene illumination, the adaptive optics may provide a uniformdistribution and during a second scene illumination the adaptive opticsmay provide a nonuniform (e.g., Gaussian, gradient, line on-line off) orsimilar illumination in response to control and/or processing circuitrysignals or commands.

In some approaches, the adaptive optics may provide different responsesaccording to the spectral content of the supplemental illumination. Forexample, the adaptive optics may provide a first distribution ofsupplemental illumination during a first period of supplementalillumination (e.g., red wavelengths) and provide a second distributionof supplemental illumination during the second period of supplementalillumination (e.g., blue wavelengths). Where the adaptive optics arewavelength selective, the distribution of supplemental illumination fora first wavelength may differ from the distribution of supplementalillumination of a second wavelength, even during a common period ofillumination. The different distributions of supplemental illuminationare not necessarily limited to different “color” bands. For example,wavelength selective optics may provide a first distribution ofsupplemental illumination for a first green wavelength and a seconddistribution of supplemental illumination for a second green wavelength.Such narrowband optics may include narrowband filters, diffractiveelements, or other wavelength selective structures or devices.

With renewed reference to FIGS. 1-3, imaging device 110 may beconfigured to detect a finite number M of spectral components whichcorrespond to M primary color components of a color gamut of the imagingdevice. Imaging device 110 may include one or more imaging pixels, eachconfigured to detect at least one of a finite number M of spectralcomponents corresponding to M primary components of a color gamut of theimaging device. For example, an imaging pixel may be configured todetect one of three spectral components corresponding to additive red,green, and blue primary color components, or to subtractive cyan,magenta, and yellow primary color components of a color gamut of theimaging device.

In the case where image sensor 120 is coupled to a spatial color filterarray, each image sensor 120 pixel senses only the particular spectralcomponent that is passed through to it by filter array. For such cases,the processing circuitry may be configured to compute additional“native” color values for the pixels by spatial interpolation ofneighboring pixel data. Any suitable color interpolation algorithms maybe deployed for this purpose. The processing circuitry may be furtherconfigured to use the native color values obtained by spatialinterpolation of pixel values for the N detected spectral components ofreceived light under the first scene illumination to provide a nativecolor rendition of the scene under the first scene illumination, and tofurther modify this native color rendition according to the values ofthe detected spectral components of received light under the secondscene illumination. Alternatively, the processing circuitry may beconfigured to modify pixel values for the N detected spectral componentsof received light under the first scene illumination according to thevalues of the detected spectral components of light transmitted from thescene under the second scene illumination. The processing circuitry maybe further configured to compute a native color rendition of the scenebased on, for example, spatial interpolation of the modified values ofthe detected spectral components of light transmitted from the sceneunder the first scene illumination.

For all cases of native color rendition (e.g., based on interpolationfollowed by modification, or vice versa), the processing circuitry maybe further configured to apply a predetermined color correction routineto the native color rendition to yield a corrected color rendition ofthe scene. The predetermined color correction routine may be based onbased on characteristics of imaging device optics (e.g., lens andfilters).

FIG. 3 shows an apparatus 300 for modifying a spectral responserepresentation of a scene obtained by an imaging device (e.g., device310). Apparatus 300 may include a supplemental illumination device(e.g., supplemental illumination device 140) and/or data processingcircuitry 370 (e.g., data and/or signal processor 170). Apparatus 300may further include a control unit 360 coupled to the supplementaldevice and/or data processing circuitry 370. The supplementalillumination device may be operable to illuminate the scene with light142 having a spectral distribution for which at least some qualifyinginformation is available (e.g., “a known spectral distribution”).

Control unit 360 may be configured to establish different types of sceneilluminations provided by the supplemental illumination device and acorresponding number of different sets of spectral components of lightto be detected by the imaging device for rendering a color image of aparticular scene. Control unit 360 may be configured to determine thedifferent types of scene illuminations and their sequence for renderinga color image of a particular scene in response to at least one ofambient light intensity, returned light characteristics, scene distance,and/or a control signal. A first of the distinct scene illuminations mayinclude ambient and/or artificial light provided by an optional firstillumination device, and a second of the distinct scene illumination mayreplace the ambient and/or artificial light with, or additionallyinclude, light having a known spectral distribution.

An apparatus, which may be used in conjunction with imaging devices toobtain color images of a scene or to modify a detected spectral powerdistribution response of the scene to light, includes a supplementalillumination device operable to illuminate the scene with light having aknown spectral distribution. The apparatus may additionally oralternatively include processing circuitry to render a color image of ascene or produce a signal indicative of a modified spectral powerdistribution response of the scene to light. The processing circuitrymay be responsive a plurality of sets of spectral components of lightreceived from the scene under a corresponding plurality of sceneilluminations including at least one illumination with light having aknown spectral distribution. The processing circuitry may render a colorimage of the scene, or modify a spectral power distribution response ofthe scene, based at least in part on the spectral components of lightreceived from the scene under two different scene illuminationsincluding one with light having a known spectral distribution.

Data processing circuitry 370 may include an interface 380 for receivingvalues of detected spectral components of light and other data and/orsignals from the imaging device and the supplemental illuminationdevice. Data processing circuitry 370 may be configured to be responsiveto the data representative of the supplemental illumination device, datarepresentative of characteristic spectral response functions Rn(λ), n=1,. . . N, corresponding to N spectral components detected by the imagingdevice, and data representative of light responses of the scene underdistinct illuminations, at least one of which includes the light havinga known spectral distribution, to produce data and/or a signalindicative of a modified spectral response representation of the scene.

Further, data processing circuitry 370 may be configured to render acolor image of a scene by processing a plurality of sets of spectralcomponents of light received from the scene under a correspondingplurality of scene illuminations. The sets may include at least a firstand a second set corresponding to first and second scene illuminationsthat include light having an unknown spectral distribution and lighthaving a known spectral distribution, respectively. The rendered colorimage of the scene may be based at least in part on the detectedspectral components of light received from the scene under the first andsecond scene illuminations, the characteristic spectral responsefunctions R(λ) of the imaging device, and the known spectraldistribution of light in the second scene illumination.

Furthermore, data processing circuitry 370 may be configured to computea color rendition (e.g., a native color rendition) of the scene inspatial dimensions based on spatial interpolation of the detectedspectral components of light received from the scene under the firstscene illumination, and to modify the rendition based at least in parton values of the second set of spectral components of lightcorresponding to the second scene illumination.

Data processing circuitry 370 may be alternatively or additionallyconfigured to modify values of the first set of spectral components oflight corresponding to the first scene illumination at least in partaccording to values of the second set of spectral components of lightcorresponding to the second scene illumination, and to compute a colorrendition of the scene in spatial dimensions based on the modifiedvalues, for example, by spatial interpolation of the modified values.

Additionally, data processing circuitry 370 may be configured to apply apredetermined color correction routine to color renditions of the scene(e.g., native or modified renditions) to yield corrected colorrenditions of the scene. The predetermined color correction routine maybe based on imaging device optics.

Data processing circuitry 370 also may be configured to estimate aspectral distribution of light having an unknown spectral distributionin the first scene illumination based at least in part on the detectedspectral components of light received from the scene under the first andsecond scene illuminations, the characteristic spectral responsefunctions R(λ) of the imaging device, and the known spectraldistribution of light in the second scene illumination, and to furthercompute a color rendition of the scene (e.g., a native color rendition)based at least in part on the detected spectral components of lightreceived from the scene under the first scene illumination, theestimated spectral distribution of such light in the first sceneillumination, and the characteristic spectral response functions R(λ) ofthe imaging device.

Methods for improving a color image or rendition of a scene may utilizeadditional information on the scene SPD developed by illuminating thescene with light having a known spectral distribution. FIGS. 6-8 showexemplary methods 600, 700 and 800 which utilize additional informationon the scene SPD developed by illuminating the scene with light having aknown spectral distribution to improve color images or scene renditions.

A method for modifying a first color image or rendition of a sceneobtained under a first scene illumination, involves detecting a spectralresponse of the scene to the light having a known spectral powerdistribution under a second scene illumination, and modifying the firstcolor image according to the detected spectral response of the scene.

FIG. 6 shows exemplary method 600 for improving a color image or scenerendition, which may have been obtained under a first sceneillumination. The first scene illumination may, for example, haveincluded light having an unknown spectral power distribution. The colorimage or rendition may be based on values of a finite number of spectralcomponents of light received from the scene as detected by an imagingsensor, which has characteristic spectral response functions fordetecting each of the finite number of spectral components. The firstscene illumination may include ambient and/or artificial light.

Method 600 involves illuminating at least a portion of the scene in asecond scene illumination with a light component having a known spectralpower distribution (610), detecting a finite number of spectralcomponents of light received from the scene under such illumination(620), and modifying the color image or rendition of the scene accordingto the detected values of the finite number of spectral components oflight received from the scene under such illumination so that the firstcolor image or rendition of the scene is improved (630). The modifiedimage or scene rendition may be based on spatial interpolation ofmodified values of the detected spectral components of light receivedfrom the scene under the first scene illumination.

The light component having the known spectral power distribution mayinclude substantially monochromatic light at one or more wavelength(s)λ_(k), k=1, . . . K. Adaptive optics may be used to direct the lightcomponent onto the portion of the scene, which, for example, maycorrespond to one or more pixels of an imaging device.

Method 600 may involve adding the light component having the knownspectral power distribution to the first scene illumination under whichthe unmodified color image was obtained, or to other sceneilluminations.

Method 600 may further involve illuminating the same or other portionsof the scene under third or more illuminations with the same or otherlight components having known spectral power distributions (612),detecting values of the finite number of spectral components of lighttransmitted by the scene under the third or more scene illuminations(620), and modifying the color image or rendition of the scene accordingto the detected values of the finite number of spectral components oflight received from the scene under one or more such illuminations(630). The number and sequence of scene illuminations may bepre-determined, or established within method 600, based onconsiderations of, for example, color image quality, and availability oflight sources, imaging equipment and computing resources. The number andsequence of illuminations may be determined iteratively, for example, byconsideration of the quality of iteratively modified color images. FIG.6 shows an optional process 602 in method 600 for establishing thedifferent types of scene illuminations, their sequence, and thecorresponding number of different sets of spectral components of lightto be detected by the imaging device for rendering an improved colorimage of a particular scene. The process may involve establishingdifferent types of scene illuminations in response to at least one ofambient light intensity, returned light characteristics, scene distance,imaging device optics, or a control signal.

Method 600 may also include applying a predetermined color correctionroutine to the modified color image or rendition of the scene to yield acorrected color rendition of the scene. The predetermined colorcorrection routine may be based on consideration of imaging deviceoptics.

Further, method 600 may also include transmitting values of the spectralcomponents of light detected by the imaging device to a data processingdevice, and at the data processing device, processing the values ofspectral components of light to generate the improved color image orrendition of the scene. The processing may involve processing liveimaging device data, or processing such data at a later time (e.g.,batch processing such data). The data processing device may be at theimaging device or at a remote location.

Methods for rendering a color image of a scene in response to control orcommand signal may involve processing a plurality of sets of spectralcomponents of light received from the scene under a correspondingplurality of scene illuminations, and rendering the color image of thescene based at least in part on the spectral components of lightreceived from the scene under two scene illuminations, at least one ofwhich includes light with the known spectral distribution FIG. 7 showsan exemplary method 700 for rendering a color image of a scene.

Method 700 includes, in response to the control signal, processing aplurality of sets of values of spectral components of light receivedfrom the scene under a corresponding plurality of scene illuminations(710). The control signal may be generated in response to at least oneof ambient light intensity, returned light characteristics, imagingdevice optics, or imaging device-scene distance. The plurality of setsof values may include at least a first set corresponding to a firstscene illumination and at least a second set corresponding to a secondscene illumination that includes light having a known spectraldistribution. Each set of values of spectral components of lightreceived from the scene may have been detected by an imaging devicehaving characteristic spectral response functions for detecting suchspectral components.

Method 700 further includes rendering the color image of the scene basedat least in part on the detected spectral components of light receivedfrom the scene under the first and second scene illuminations, thecharacteristic spectral response functions of the imaging device, andthe known spectral distribution of light in the second sceneillumination (720).

In method 700, rendering the color image (720) may include rendering afirst color image of the scene in spatial dimensions based on spatialinterpolation of the detected spectral components of light received fromthe scene under the first scene illumination. Further, rendering thecolor image (720) may involve modifying values of the first set ofspectral components of light corresponding to the first sceneillumination according to at least values of the second set of spectralcomponents of light corresponding to the second scene illumination,and/or computing a native color rendition of the scene based on themodified values of the detected spectral components of light receivedfrom the scene under the first scene illumination.

Additionally or alternatively, in method 700, rendering the color image(720) may include estimating a spectral distribution of light in thefirst scene illumination based at least in part on the detected spectralcomponents of light received from the scene under the first sceneillumination and the characteristic spectral response functions of theimaging device, and further include modifying the estimated spectraldistribution based at least in part on the detected spectral componentsof light received from the scene under the second scene illumination,the characteristic spectral response functions of the imaging device,and the known spectral distribution of light in the second sceneillumination. Further, method 700 may include computing a native colorrendition of the scene based at least in part on the modified estimatedspectral distribution.

Method 700, like method 600, may further include applying apredetermined color correction routine to the color image (e.g., nativeor modified color image) to yield a corrected color image. Further, likein method 600, processing a plurality of sets of values (710) in method700 may occur at least in part at the imaging device and/or a remotelocation with respect to the imaging device. Further, processing aplurality of sets of values (710) may involve processing live imagingdevice data and/or processing such at data at a later time.

Methods for adjusting a characteristic spectral response function of animaging device while “in use” to image a scene, may involve recording aspectral response of the imaging device to illumination of at least aportion of the scene with a light component having a known spectralpower distribution and adjusting the characteristic spectral responsefunction of the in use imaging device according to the recorded spectralresponse.

Method 800 includes, in a second scene illumination, illuminating atleast a portion of the scene with at least a light component having aknown spectral power distribution (810). The light component having aknown spectral power distribution may, for example, be substantiallymonochromatic light at one or more wavelength(s) λ_(k), k=1, . . . K.The second scene illumination may precede, succeed, add to, or modifythe first scene illumination. Method 800 further includes recording aspectral response of the imaging device to light received from the sceneunder the second scene illumination (820), and adjusting acharacteristic spectral response function of the imaging deviceaccording to the recorded spectral response of the imaging device tolight received from the scene under the second scene illumination (830).

In method 800, adjusting the characteristic spectral response functionof the imaging device (830) may include modifying values of the spectralcomponents detected by the imaging device under the first sceneillumination according to values of the spectral components detected bythe imaging device under the second scene illumination. Alternatively oradditionally, adjusting the characteristic spectral response function ofthe imaging device (830) may include rendering a first image of thescene based on values of the spectral components detected by the imagingdevice under the first scene illumination, and modifying the first imageat least in part according to the values of the spectral componentsdetected by the imaging device under the second scene illumination.

Method 800, like method 700, may be activated in response to the controlsignal, which is generated, for example, in response to at least one ofambient light intensity, returned light characteristics, imaging deviceoptics, or imaging device-scene distance. Further, like in methods 600and 700, method 800 may be implemented at least in part at the imagingdevice and/or a remote location with respect to the imaging device, andmay involve processing live imaging device data and/or processing suchat data at a later time.

The foregoing detailed description has set forth various embodiments ofthe devices and/or processes via the use of block diagrams, flowcharts,and/or examples. Insofar as such block diagrams, flowcharts, and/orexamples contain one or more functions and/or operations, it will beunderstood by those within the art that each function and/or operationwithin such block diagrams, flowcharts, or examples can be implemented,individually and/or collectively, by a wide range of hardware, software,firmware, or virtually any combination thereof. In one embodiment,several portions of the subject matter described herein may beimplemented via Application Specific Integrated Circuits (ASICs), FieldProgrammable Gate Arrays (FPGAs), digital signal processors (DSPs), orother integrated formats. However, those skilled in the art willrecognize that some aspects of the embodiments disclosed herein, inwhole or in part, can be equivalently implemented in integratedcircuits, as one or more computer programs running on one or morecomputers (e.g., as one or more programs running on one or more computersystems), as one or more programs running on one or more processors(e.g., as one or more programs running on one or more microprocessors),as firmware, or as virtually any combination thereof, and that designingthe circuitry and/or writing the code for the software and or firmwarewould be well within the skill of one of skill in the art in light ofthis disclosure. In addition, those skilled in the art will appreciatethat the mechanisms of the subject matter described herein are capableof being distributed as a program product in a variety of forms, andthat an illustrative embodiment of the subject matter described hereinapplies regardless of the particular type of signal bearing medium usedto actually carry out the distribution. Examples of a signal bearingmedium include, but are not limited to, the following: a recordable typemedium such as a floppy disk, a hard disk drive, a Compact Disc (CD), aDigital Video Disk (DVD), a digital tape, a computer memory, etc.; and atransmission type medium such as a digital and/or an analogcommunication medium (e.g., a fiber optic cable, a waveguide, a wiredcommunications link, a wireless communication link (e.g., transmitter,receiver, transmission logic, reception logic, etc.), etc.).

Further, those skilled in the art will recognize that at least a portionof the devices and/or processes described herein can be integrated intoan image processing system. Those having skill in the art will recognizethat a typical image processing system generally includes one or more ofa system unit housing, a video display device, memory such as volatileor non-volatile memory, processors such as microprocessors or digitalsignal processors, computational entities such as operating systems,drivers, applications programs, one or more interaction devices (e.g., atouch pad, a touch screen, an antenna, etc.), control systems includingfeedback loops and control motors (e.g., feedback for sensing lensposition and/or velocity; control motors for moving/distorting lenses togive desired focuses). An image processing system may be implementedutilizing suitable commercially available components, such as thosetypically found in digital still systems and/or digital motion systems.

While various aspects and embodiments have been disclosed herein, otheraspects and embodiments will be apparent to those skilled in the art.The various aspects and embodiments disclosed herein are for purposes ofillustration and are not intended to be limiting, with the true scopeand spirit being indicated by the following claims.

The invention claimed is:
 1. A system for acquiring spectralcharacteristics of a scene under a first scene illumination, whereinlight received from the scene has a spectral power distribution (SPD)response characteristic of the scene, the system comprising: an imagingdevice configured to detect a finite number (N) of spectral componentsof light received from the scene, the imaging device havingcharacteristic spectral response functions R_(n)(λ), n=1, . . . N, fordetecting such spectral components and wherein: the image devicecomprises one or more imaging pixels, each configured to detect at leastone of a finite number M of spectral components corresponding to Mprimary components of a color gamut of the imaging device; and animaging pixel is configured to detect one of three spectral componentscorresponding to subtractive cyan, magenta, and yellow primary colorcomponents of a color gamut of the imaging device; a supplementalillumination device operable to provide a second scene illumination withlight having a known spectral distribution; and processing circuitrycoupled to the imaging device and responsive at least to: the detectedspectral components of light received from the scene under the first andsecond scene illuminations; the characteristic spectral responsefunctions of the imaging device; and the known spectral distribution ofthe light provided by the supplemental illumination device, to produce asignal indicative of the SPD response characteristic of the scene. 2.The system of claim 1, wherein the first scene illumination includesambient and/or artificial light provided by an artificial lighting unit,and wherein the supplemental illumination device is operable so that thesecond scene illumination additionally includes the light with the knownspectral distribution.
 3. The system of claim 1, wherein the first sceneillumination includes ambient and/or artificial light provided by anartificial lighting unit, and wherein the supplemental illuminationdevice is operable so that in the second scene illumination the lightwith the known spectral distribution replaces the ambient and/orartificial light of the first scene illumination.
 4. The system of claim1, wherein the supplemental illumination device comprises an array ofone or more light emitting diodes.
 5. The system of claim 1, wherein thesupplemental illumination device comprises an array of one or moresubstantially monochromatic light sources.
 6. The system of claim 1,wherein the supplemental illumination device comprises an array of oneor more light sources emitting light at the one or more wavelength(s)λ_(k), k=1, . . . K, respectively.
 7. The system of claim 6, wherein theone or more light sources emitting light at the one or morewavelength(s) λ_(k), k=1, . . . K, correspond to one or more differentcolors R, G, B, Y, C, M, . . . , respectively.
 8. The system of claim 1,wherein the supplemental illumination device comprises a CIE-basedfilter set.
 9. The system of claim 1, wherein the supplementalillumination device comprises an array of lenses or microlensesconfigured to direct light having the known spectral distribution on toat least a portion of the scene.
 10. The system of claim 1, wherein theimaging device comprises at least one of a CMOS image sensor or a CCDimage sensor.
 11. The system of claim 1, wherein the imaging device isconfigured to detect a finite number M of spectral componentscorresponding to M primary components of a color gamut of the imagingdevice.
 12. The system of claim 1, wherein an imaging pixel isconfigured to detect one of three spectral components corresponding toadditive red, green, and blue primary color components of a color gamutof the imaging device.
 13. The system of claim 1, wherein at least twoof the imaging device's characteristic spectral response functionsR_(n)(λ), n=1, . . . N, which respectively correspond to the N detectedspectral components, substantially overlap.
 14. The system of claim 1,wherein the processing circuitry is configured to compute a native colorrendition of the scene based on spatial interpolation of the N detectedspectral components of received light under the first sceneillumination.
 15. The system of claim 14, wherein the processingcircuitry is configured to modify the native color rendition of thescene based on spatial interpolation the N detected spectral componentsof received light under the first scene illumination according to thevalues of the detected spectral components of received light under thesecond scene illumination.
 16. The system of claim 14, whereinprocessing circuitry is configured to apply a predetermined colorcorrection routine to the native color rendition of the scene to yield acorrected color rendition of the scene, wherein the predetermined colorcorrection routine is based on imaging device optics.
 17. The system ofclaim 1, wherein the processing circuitry is configured to modify valuesof the detected spectral components of received light under the firstscene illumination according to the values of the detected spectralcomponents of light transmitted from the scene under the second sceneillumination.
 18. The system of claim 17, wherein the processingcircuitry is further configured to compute a native color rendition ofthe scene based on spatial interpolation of the modified values of thedetected spectral components of light transmitted from the scene underthe first scene illumination.
 19. The system of claim 18, wherein theprocessing circuitry is further configured to apply a predeterminedcolor correction routine to the native color rendition of the scene toyield a corrected color rendition of the scene, wherein thepredetermined color correction routine is based on imaging deviceoptics.
 20. The system of claim 1, wherein at least a portion of theprocessing circuitry is disposed at a remote location with respect tothe imaging device.
 21. The system of claim 1, wherein the processingcircuitry is configured to batch process imaging device data.
 22. Anapparatus, comprising: a supplemental illumination device operable toilluminate a scene with light having a known spectral distribution; andone or more imaging pixels, each configured to detect at least one of afinite number M of spectral components corresponding to M primarycomponents of a color gamut of the apparatus, wherein each of the one ormore imaging pixels are configured to detect one of three spectralcomponents corresponding to subtractive cyan, magenta, and yellowprimary color components of the color gamut; processing circuitryresponsive at least to: receive, from the one or more imaging pixels,data representative of a spectral power distribution from the sceneunder a first scene illumination; and receive, from the one or moreimaging pixels, data representative of the known spectral distributionfrom the scene under a second scene illumination including thesupplemental illumination; and produce a signal indicative of a modifiedspectral power distribution of the scene under a first sceneillumination and the second scene illumination.
 23. The apparatus ofclaim 22, wherein the light under a first scene illumination comprisesambient and/or artificial light, and wherein the supplementalillumination device is operable so that the second scene illuminationadditionally includes the light with the known spectral distribution.24. The apparatus of claim 22, wherein the light under a first sceneillumination comprises ambient and/or artificial light, and wherein thesupplemental illumination device is operable so that in the second sceneillumination the light with the known spectral distribution the lightreplaces the ambient and/or artificial light of the first sceneillumination.
 25. The apparatus of claim 22, wherein the supplementalillumination device comprises an array of one or more light emittingdiodes.
 26. The apparatus of claim 22, wherein the supplementalillumination device comprises an array of one or more substantiallymonochromatic light sources.
 27. The apparatus of claim 22, wherein thesupplemental illumination device comprises an array of one or more lightsources emitting light at the one or more wavelength(s) λ_(k), k=1, . .. K, respectively.
 28. The apparatus of claim 27, wherein the one ormore light sources emitting light at the one or more wavelength(s)λ_(k), k=1, . . . K, correspond to one or more different colors R, G, B,Y, C, M, . . . , respectively.
 29. The apparatus of claim 22, whereinthe supplemental illumination device comprises a CIE-based filter set.30. The apparatus of claim 22, wherein the supplemental illuminationdevice comprises an array of lenses or microlenses configured to directlight having the known spectral distribution onto at least a portion ofthe scene.
 31. The apparatus of claim 22, further comprising a controlunit coupled to the supplemental illumination device, wherein thecontrol unit is configured to establish different types of sceneilluminations for rendering a color image of a particular scene.
 32. Theapparatus of claim 22, wherein the data processing device furthercomprises an interface for receiving the representative data.